Study Island
            Krise, Scott - Wed 09/29/2010   

            Benchmark Analysis
                  Program: 11th Grade - PA Standards Mastery and PSSA 
Preparation
                  Benchmark: PSSA Reading Benchmark 1


             Correlation/Predictive Validity Analysis

            The performance level cut scores have been correlated to PSSA 
            scores. The correlation was done by analyzing PSSA scores vs. 
            benchmark scores for 132 students in the 2008-2009 school year. All 
            students took this benchmark within 30 days of the actual PSSA. 

                  Benchmark Performance Level Cut Scores
                  (based on 100 point test)
                   AdvancedProficientBasicBelow Basic
                  Raw Score100-8685.99-69.3369.32-5655.99-0
                  Scaled Score2546-14921491-12571256-11121111-700


                  Students Tested:
                  (during PSSA testing window)132
                  Correlation Coefficient:
                  (Benchmark vs. PSSA)0.99

            Correlation Coefficient:
            The correlation coefficient indicates the strength of the 
            relationship between the performance levels from this benchmark test 
            to the actual PSSA performance levels. A correlation coefficient of 
            1 implies a perfect correlation between this benchmark and the 
            actual PSSA. A correlation coefficient of 0 implies no correlation. 
            Given the number of students involved in this study, the correlation 
            is not likely to be perfect (1). Any correlation coefficient above 
            0.7 is an acceptable correlation. 

            Test Equivalence:
            We ensure all four benchmarks are equivalent by correlating each 
            test to the PSSA. Since all four benchmark tests are equivalent to 
            the PSSA, the benchmark tests are also correlated to each other.


             Test Reliability Analysis

            Below are the reliability statistics for this benchmark test.

                  Reliability Statistics
                  (based on 100 point test)
                  Students Tested:3,973
                  Test Reliability:
                  (Coefficient Alpha)0.87
                  Standard Error of Measurement:7.33
                  Statewide Mean Score:
                  (during PSSA testing window)68.92

            Reliability:
            The reliability for this benchmark test is 0.87, indicating highly 
            consistent test scores.

                        Reliability Rule of Thumb
                        > 0.8 Good> 0.7 Acceptable> 0.6 Questionable> 0.5 Poor< 
                        0.5 Unacceptable

                  Source: George and Mallery (2003)

            The reliability coefficient is a “unitless” index, which can be 
            compared from test to test. Although a number of reliability indices 
            exist, a frequently reported index for achievement tests is 
            Coefficient Alpha (shown above). Alpha indicates the internal 
            consistency over the answers to a set of questions measuring an 
            underlying achievement. From this perspective, Alpha can be thought 
            of as the correlation between scores if the students could be tested 
            twice with the same instrument without the second testing being 
            affected by the first. It can also be conceptualized as the extent 
            to which an exchangeable set of items from the same domain would 
            result in similar ordering of students. 

            Standard Error of Measurement:
            The standard error of measurement is another indicator of precision. 
            If everyone being tested had the same true score, there would still 
            be some variation in observed scores due to imperfections in the 
            measurement process, such as random differences in attention during 
            instruction or concentration during testing. The standard error is 
            defined as the standard deviation of the distribution of observed 
            scores for students with identical true scores. Because the standard 
            error of measurement is an index of the random variability in test 
            scores in actual score units, it represents important information 
            for test score users. 



      Copyright © 2010 Study Island - All rights reserved. 
